1. Field of the Invention
The present invention relates to a method for the detection and removal of line-like defects from an image. The invention relates to a process of detecting the defects using mathematical procedures and then correcting the detected defects.
2. Background of the Art
As a result of repeated handling or accident, old photographs often develop scratches. They also acquire defects resulting from material failure with age, such as cracking of surface coatings or laminations placed over the surface of the picture. The photograph may also become creased through careless handling, leading to a surface pattern resembling a scratch. Such scratch patterns are in general not simple straight lines. Instead they can be quite complex, having curved as well as straight sections with intersections and line junctions. Since the advent of inexpensive and high quality scanners, many old photographs are being digitized. Such images have high sentimental or historical value and there is a strong desire and need to digitally eliminate defects such as scratches from the images.
Much effort has been expended on restoration of cinema films that have undergone damage. An overview of the area can be found in P. Schallauer, A. Pinz and W. Haas, “Automatic restoration algorithms for 35 mm film”, Videre, vol. 1, no.3, Summer 1999. This is an electronic journal available on-line from http://mitpress.mit.edu/e-journals/Videre/. Film scratches are usually substantially straight and lie along the film direction. This orientation of the scratches occurs because the scratches are usually caused by the projector transport mechanism, which lies consistently parallel to the film sprockets, and because scratches caused by trapped dirt during development can move over a few percent of the film width over several frames, again along the path of film movement during development. Compared to the arbitrary scratches that are found in still photographic prints, the scratching of motion picture film is therefore a very restricted form of scratch. Despite this, the authors state that “from an algorithmic point of view, removing defects is an easy task in comparison to detecting them”. This position attests to two things: first, detecting even a very specific type of scratch is difficult and, second, correction of films is relatively easy because content can be taken from frames prior to and after the defect. There is, however, no such source of data for a still photo.
In digital image editing software such as, for instance, Paint Shop Pro (Jasc Software, Inc., 7905 Fuller Road, Eden Prairie, Minn., 55344), it is customary to provide a clone tool to repair image defects. This tool is a brush that picks up an undamaged region of the image and allows it to be painted over the damaged portion. Such a tool is effective, but requires considerable skill to produce a natural and seamless result. Moreover, the tool is not helpful when the image contains no area with undamaged content corresponding to that of the damaged area. Other software often provides a “dust and scratch removal” filter where defects are removed with a non-linear filter, such as a median filter. However, since every pixel in the image—not just scratch pixels—can potentially be modified, this technique is of very limited usefulness. For this reason, a number of proposals have been made for improved methods of scratch correction. For example, in M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image Inpainting”, Preprint 1655, Institute for Mathematics and its Applications, University of Minnesota, December 1999 there is described a method of filling-in scratches, which is done in such a way that isophote lines arriving at the region boundaries are completed inside. Though the method is described as fast, because it is iterative, securing the highest quality results is slow. In a paper entitled “Combining Frequency and Spatial Domain Information for Fast Interactive Image Noise Removal”, A. N. Hirani and T. Totsuka, Proceedings of SIGGRAPH 96, 269–276 (1996) report a method of repairing scratches in textured areas in which the user indicates a region of source texture for patching that is applied to the defect by means of frequency domain analysis. This method can be very effective in some cases but can fail in others. Neither of these improvements provide any way to identify the scratch. In fact, both authors assume that the scratch has previously been defined somehow, presumably by hand.
However, when scratches or other defects are small and plentiful, defining the scratches prior to correcting them becomes a main objective and content of labor. For instance, the work of marking every section of every scratch is roughly equivalent to cloning or painting over the scratch in the first place, so little is gained by subsequent automation. Automatic definition of generalized scratches is not at a satisfactory level. Often knowing whether something is a scratch or not depends on understanding the content of the image, something beyond the ability of current methods dealing with arbitrary images. A need therefore exists for an assisted method of scratch removal that will allow a user to define precisely the defects in an image without needing the labor of tracing each individually and then to perform a correction of the so-defined scratches.